The Town Quay scheme by developer Nicholas James Group will be spread across three identical 25-storey residential buildings and a seven-storey block nearby.
The project also includes a 9-storey building with two connected elements on the existing pier comprising a hotel with 128 bedrooms and 18 serviced apartments, spa, restaurants and conferencing facilities.
The large footprint building will include two floors of parking below and a helipad on the roof.
It will include a new red-jet passenger terminal and cafe at ground floor level with associated new ferry berths, following pier strengthening works.
The new development, which also includes a 300-berth marina, will open up the prominent waterfront site, which has remained largely inaccessible to the public for many years.
The project is expected to cost around £230m to build and will be funded by private equity and sale of the flats. It will be delivered in two phases over four years.